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@ Biopotential digital controller for music and video applications. 



@ There is disclosed apparatus end methods for 
sensing electrical signals generated in a living body 
and mapping these signals to MIDI code. The ap- 
paratus senses electrical signals generated in the 
body through electrodes on the skin or implanted in 
various areas of the body. The electrical signals are 
then amplified, filtered and converted to digital data. 
This digital data Is analyzed to determine character- 
istics of the electrical signals such as amplitude, 
frequency, change in amplitude or change in fre- 
quency. These characteristics are then mapped to 

a MIDI code or other code to control the generation of 
musical notes or video display information. There is 
Qalso disclosed a method and apparatus for determin- 
Qjing the alpha wave or other content of brain waves 
^by taking a fast fourier transform of the brain wave 
^.and determining the coefficient for the fourier fre- 
O^iuency component at the frequency of desired fre- 
COquencies. Further, a method of altering MIDI code 
Q generated from biopotentials is disclosed where the 
effect of musical notes generated by the MIDI code 
2^ on the biopotentials causes changes in the MIDI 
code. A similar method is also disclosed wherein 
MIDI coda received from another machine Is 



changed based upon mapping of biopotentials from 
the body of a subject. 
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BIOPOTENTIAL DIGITAU CONTROLLER FOR MUSIC AND VIDEO APPLICATIONS 



Background of the Invention 

The invention relates to the field of connputer 
generated music. Specifically, the invention relates 
to the field of sensing electrical signals generated 
in a human body and mapping those signals Into 
MIDI code which controls the generation of musical 
notes. 

Various methods of collecting biological data 
by sensing electrical signals generated by a living 
body and converting these signals to audible or 
visual manifestations have long been in use. These 
systems have clinical and psychotherapeutic ap- 
plications. Examples of such systems are found in 
US Patent 4.138.684 to Scattergood et al.. and US 
Patent 4.170.225 to Criglar et ai. In Scattergood et 
al., an electromyographic biofeedback system is 
disclosed. A transducer is used to sense electrical 
signals from the body, and these signals are am- 
plified by a differential amplifier to eliminate com- 
mon mode noise. After further signal processing, 
and averaging in a third order averaging filter, the 
output signal from the averaging filter Is used to 
control a current controlled oscillator which pro- 
vides a series of audible pulses at a rate which 
varies linearly with the value of the input voltage 
detected by the transducer. The system is useful in 
detecting small muscle movements to indicate 
progress in rehabilitation of patients. 

The Criglar reference teaches a portable 
biofeedback device for monitoring electrical signals 
produced by a muscle activity. A series of filters 
and preamplifiers eliminate noise components and 
unwanted frequency components such that only 
signals in a certain frequency range are amplified. 
The signals monitored are those produced by mus- 
cle activity. Both visual and audio feedback Is 
generated for use by the person being monitored. 
\ Another class of prior art devices senses brain 
wave activity and converts the brain waves or com- 
ponents thereof to visible or audible feedback sig- 
nals. One such device is disclosed in US Patent 
4.228.807 to Yagi et al. Yagi teaches a biofeedback 
device which senses brain waves and separately 
extracts the alpha wave component and the muscle 
component included in the brain wave. Yagi teach- 
es generating auditory feedback which allows the 
user to recognize accurately the appearance of the 
alpha wave component in the brain wave, thereby 
indicating the state of relaxation of the subject 
Audible sounds are also generated for the muscle 
component in the brain wave which allows the user 
to know when tension in the subject's mental state 
exists. 



Another such system Is taught by Shiga in US 
Patent 4.334.545. Shiga teaches a system to detect 
the brain wave components of the subject in low 
and high frequency bands. The low frequency band 
s has a frequency range approximately the same as 
the alpha wave component generated by the brain, 
and the higher frequency range has a frequency 
which is considerably higher than the highest fre- 
quency of the brain alpha waves. Audible re- 

70 sponses are generated. Indications are generated 
when the detected high frequency components 
have an amplitude less than a level indicative that 
the subject Is not In a state of relaxation and when 
the detected low frequency component has an am- 

/5 plitude greater than a first predetermined value and 
less than a second predetermined value. When the 
low frequency amplitude is between these two pre- 
determined values, the subject is generating alpha 
waves and is in a state of relaxation. 

30 Another alpha wave sensing system is taught 

by Shiga in US Patent 4.354.505. In this reference, 
Shiga teaches a self-training biofeedback system 
which has a binary counter. The counter counts the 
intervals where alpha wave components are 

25 present which represents the relaxation period. A 
visual indicator connected to the counter provides 
the person being trained with a display of the 
measured time period when alpha waves are being 
generated so that the depth of relaxation can be 

30 visualized. 

A method for generating sounds from brain 
waves is taught In US Patent 4.454.886 to Lee. In 
this system, the brain wave signal derived from an 
electroencephalograph is converted in real time 

35 from an analog to a digital signal and stored in a 
memory. Another circuit then detects points in time 
along this EEG signal which define segments of the 
signal. The signals in the memory are then read 
out in segments with the segment signals being 

40 repeatedly read out n number of times at a rate n 
times faster than the input rate to the memory. This 
generates a train of signals which are high fre- 
quency replicas of the signal segment. The seg- 
ments are stored in a FIFO memory such that as 

45 the first segment to be stored is the first segment 
to be read out of memory, and the second seg- 
ment to be stored in the memory is the second 
segment to be read out of the memory. 

Biological activities manifested in electrical sig- 

50 nals occurring in an individual, a couple, or a group 
of persons manifested In electrical signals are 
sensed and utilized in a system taught by Fehmi et 
al. in US Patent 4.031.883. Multiple electrical con- 
tacts are applied to the scalp or body of the 
individual or all the people in the group. The au- 
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dible sensory feedback signals generated from the 
electrical signals detected by these electrical con- 
tacts simultaneously communicate to the subject or 
subjects ongoing fluctuations in frequency, ampli- 
tude and phase unity among two or more biological 
rhythms occurring within the individual or group of 
individuals. These auditory signals may be used to 
establish awareness and control over the functions 
of the brain which are being monitored, or as an 
aid to meditation, or as an aid to learning to flexibly 
and easily establish or dissolve various states of 
attention • and consciousness associated with the 
characteristics of the signals being sensed. 

None of the above-described prior art systems 
generate musical notes In response to the sensed 
signals, and none generates musical data In the 
MIDI interface cede. 

The only prior art system of which the Ap- 
plicants are aware which generates musical notes 
in response to biological information is a system 
developed at Michigan State University by John F. 
Holland, Professor of Urology. In this system, on 
information and belief, a chemical analysis is made 
of urine samples. The chemical assay machine 
generates an analog output signal representing the 
components in the urine. Such an assay may be 
made by a liquid chromatography system. The 
analog output signal is coupled to a computer via 
an anaiog-to-digital converter. A musical note Is 
assigned to each peak in the analog signal by IBM 
personal computer which is programmed to convert 
the digital stream of data to MIDI format command 
words. This MIDI data may be sent to a musical 
Instrument for conversion to musical notes which 
are audible to the users of the system. The com- 
puter may be programmed to play a tune such as 
the "Star Spangled Banner" for a normal pattern of 
urine components. 

None these prior art systems teaches a real 
time system which can sense electrical signals 
generated in the body either in muscle tissue or In 
the brain and convert them in real time to MIDI 
format command words which are then immedi- 
ately converted to musical notes. Such a system 
would have numerous applications in such widely 
diverse fields as clinical analysis, biofeedback, 
games, musical instruments, live performances by 
artists, and many other applications. 



Summary of the Invention 

According to the teachings of the invention, 
musical notes are generated in real time from elec- 
trical signals generated in a living body which are 
collected and processed In real time. A system is 
taught herein which includes transducers for sens- 
ing one or morj3 biologically generated electrical 



signals present in a living body. These transducers 
are coupled to conversion circuitry for converting 
the analog electrical signals to a stream of digital 
data. The stream of digital data Is received by a 
s mapping unit which extracts certain information 
from the stream of digital data in real time and 
maps these characteristics into a second stream of 
digital data having a format capable of controlling 
an output device such as a music synthesizer, a 

70 video game, or some other type of device. 

In the preferred embodiment, the mapping cir- 
cuitry is a computer which is programmed to de- 
rive certain characteristics of the biologically gen- 
erated signals (sometimes hereafter referred to as 

15 biopotentials), and map these characteristics into 
one or more parameters of the MIDI interface. A 
biopotential is an electrical signal generated by 
living bodies. In the preferred embodiment, the 
conversion means converts the amplitude of the 

20 electrical signals generated in the body to digital 
words which define the amplitude during a particu- 
lar sample period. A computer then maps these 
amplitude words to corresponding note numbers in 
the MIDI interface code. The note numbers cor- 

25 respond to frequencies of notes to be generated on 
a music synthesizer serving as the output device. 
The amplitude words are then converted to "note- 
on" and "note-off" MIDI interface format code se- 
quences or data packets. These MIDI data packets 

00 are then sent to the synthesizer which generates 
musical notes having frequencies which are 
mapped in response to the intensity of the elec- 
trical signals generated in the living body. 

The invention is not limited to the particular 

35 mapping scheme described above for the preferred 
embodiment. Many different applications for the 
invention are possible, and each of these applica- 
tions may have peculiar requirements for the func- 
tion to be performed by the conversion circuitry 

40 and many different mapping functions are possible. 
For example, In the video area, the invention may 
find application In video games, computer graphics, 
and teaching aids. In the music area, the invention 
may find application in the performance arts and in 

45 professional clinical equipment for use in the of- 
fices of doctors, psychologists and psychiatrists. 
Other subareas within the music field which may 
have applications for the invention are consumer 
products and medical products. Consumer pro- 

50 ducts which may use the invention Include music 
toys, amateur instruments, relaxation devices, 
mood makers, dream manipulation devices, pet 
music devices, baby music devices, and group 
music devices. In the medical arts, the invention 

55 may find application in music therapy, orthopedic 
treatment and rehabilitation, paralysis rehabilitation 
and sleep dis order therapies or instruments for 
aiding sleep. In the mechanical arts, the invention 
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may find application in radio controlled toys, pros- 
thetics and home appliances. 

Accordingly, the invention contemplates a high 
degree of flexibility in the particular functions per- 
formed by the conversion and computer circuitry. 
For example, the conversion circuitry may Include 
several channels for gathering information in dif- 
ferent frequency bands from signals generated by 
multiple transducers placed at different locations on 
the body. The signals in the various frequency 
bands may be converted to digital signals which 
are then transmitted to the computer. The com- 
puter may then analyze these signals for various 
parameters such as the frequency of the signal, the 
intensity of the signal, the change in the frequency 
of the signal or the change in the Intensity. These 
characteristics of the signals in each channel may 
be mapped to any of the various parameters which 
may be controlled in a synthesizer or other output 
device via a standard interface such as MIDI. For 
example, if a synthesizer is used for the output 
device, the standard MIDI interface may be used 
for mapping purposes such as changes in fre- 
quency may be mapped to program number or 
harmonic content. The MIDI interface for synthesiz- 
ers and the SMPTE interface for video devices 
allow control of many characteristics of the output 
device. These characteristics control the qualities 
of the output manifestation generated by the de- 
vice. Any combination of mapping from the char- 
acteristics of the biologically generated signal to 
the parameters which may be controlled by the 
interface for the output device which is suitable for 
the particular application will be in accordance with 
the teachings of the invention. 



Brief Description of the Drawings 

Rgure 1 is a block diagram of an embodi- 
ment of the invention using a music synthesizer as 
an output device. 

Rgure 2 shows a block diagram and applica- 
tion of the teachings of the invention to a video 
game application. 

Rgure 3 is a flow diagram of a process of 
collecting electrical signals from a living body and 
analyzing these signals for certain characteristics 
and generating output manifestations in real time 
based upon these characteristics. 

Rgure 4 shows a block diagram of the pre- 
ferred embodiment of the invention in the form of a 
system which converts intensity of biological elec- 
trical signals to note frequency of musical notes 
generated in real time by a music synthesizer 
output device. 



Rgure 5 shows a flow chart of the preferred 
embodiment of the process of the invention show- 
ing a mapping function from electrical signal inten- 
sity to note frequency for real time generation of 
5 musical notes from biological electrical signals. 

Figure 6 is a flow chart of the method used 
to map the alpha wave component of EEG signals 
to MIDI code. 

Rgure 7 Is a flow diagram of the mapping 
70 process for mapping EKQ signal intensity to MIDI 
code for a MIDI drum machine. 

Rgure 8 is a flow chart representing a pro- 
cess of altering MIDI code which was generated in 
response to biopotentials by reevaluating the 
IS biopotentials after the body has had a chance to 
react to the musical notes generated in response to 
the biopotentials as they were first sensed. 

Figure 9 shows a flow diagram of a process 
for receiving MIDI code from another machine and 
20 evaluating for biopotentials from a body and then 
mapping the biopotentials to corresponding modi- 
fications in the Incoming MIDI code and then re- 
outputting the modified MIDI code. 

Figure 10 shows a block diagram of an em- 
25 bodiment of a system according to the teachings of 
the invention which senses four different kinds of 
biopotentials and receives MIDI code from another 
machine and for generating MIDI code in response 
to the biopotentials. 

30 

Detailed Description of the Preferred Embodiment 

Referring to Figure 1 there is shown a block 

35 diagram of the preferred embodiment of a system 
according to the teachings of the invention. A hu- 
man subject 10 serves as the source for electrical 
signals which are generated in the body of the 
human subject. Figure 1 shows several transducers 

40 12, 14 and 16 attached to the head of the subject. 
These transducers sense brain wave signals from 
the human subject 10. and output signals on the 
signal lines 18. 20 and 22 to a preamplifier 24. 

According to the teachings of the invention, the 

45 electrical signals from the human subject which are 
sensed by the transducers may be any type of 
signal including EEG brain wave signals. EKG 
(electrocardiogram) signals emanating from mus- 
cles of the heart or EMQ signals emanating from 

50 muscles in other parts of the body, such as the 
eye. The positions of the transducers 12. 14 and 
16 may be anywhere on the body to detect muscle 
movements or other electrical activity in the body. 
Further, although a human subject is shown, the 

55 invention is equally applicable to the bodies of 
animals and even to plants if electrical activity is 
present in a particular form of plant. 

Another place where the transducer electrodes 
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12,14 and 1 6 may be placed on the body is in tho 
vicinity of tha eye« The eyeball is essentially a D.C. 
dipole and generates a O.C. signal which has a 
polarity related to the position of the eye. When 
electrodes are placed on either side of the eye, this 
polarity may be detected. This D.C. signat be* 
comes more negative relative to one electrode 
when the eye shifts in one direction and becomes 
more positive when the eye shifts in the other 
direction. In this way, eye movements shifting the 
eye left and right may be detected. With different 
placement of the electrodes about the eye, other 
eye movements can also be detected. Eye dipole 
signals may hereafter be referred to as EOG 
(electrooculogram) signals. 

Other possible placements of the electrodes 
12. 14 and 16 may detect EGG (electrogastrogram) 
signals. 

Although only three transducers are shown in 
Figure 1 . it is also within teachings of the invention 
to use multiple sets of transducers . Generally. It is 
preferable to use three transducers {i.e., elec- 
trodes) to sense electrical signals generated in the 
brain or in muscles such that common mode noise 
can be rejected. Having three transducers allows 
the use of one common electrode and two sensing 
electrodes. Assuming electrode 14 Is the common 
electrode in the embodiment shown In Figure 1, 
common mode signals would be signals which 
simultaneously appear between electrodes 12 and 
14 and between electrodes 16 and 14. When these 
common mode signals are removed, the differential 
mode signals are left which are the signals which 
are not common as between the aforementioned 
electrode pairs. It is these differential mode signals 
which represent the muscle or brain wave activity 
of interest. 

Removal of the common mode signals and 
amplification of the difference mode signals is per- 
formed by the preamplifier 24. The preamplifier 24 
should be a differential amplifier with a high com- 
mon mode rejection ratio and low noise generation 
internally. Common mode rejection of 120 decibeis 
rejection is typical but not critical. As long as a 
usable slgnal-to-notse ratio can be obtained, any 
performance figures of merit wilf suffice. 

The output of the preamplifier 24 is coupled 
through a filtering circuit 26 to the analog Input of 
an analog-to-dlgltal conversion circuit 28. The pur- 
pose of the filtering circuit Is to remove any 60 
cycle noise which is inductively picked up by the 
electrodes or their connecting wires or which is 
present In the output of the preamplifier 24 on the 
signal line 30. 

The anatog-to-digita! conversion circuit 28 sam- 
ples the output of the filtering circuit 26 on line 32 
at a sampling rate sufficiently high to avoid al- 
iasing. The A/D converter 28 generates a stream of 



digital data on bus 34. The sampling rate of the 
analog-to*digital conversion circuit 28 should be at 
least twice as high as the fundamental component 
of the frequency of interest to avoid aliasing prob- 

s lems according to the Nyquist criteria. 

in more complex embodiments where multiple 
filter passbands are used, separate analog-to-digital 
(sometimes hereafter referred to as A/D) conver- 
sion circuits for each channel may be used. Since 

70 each passband filter puts out a different frequency 
or band of frequencies, separate sample rates must 
be used on each channel to satisfy the Nyquist 
criteria. Those skilled in the art will appreciate how 
the dominant frequencies in each passband should 

IS be sampled to get an accurate digital representa- 
tion of the analog signal in each passband. 

In the preferred embodiment, the analog-to- 
digital conversion circuit 28 serves to sample the 
amplitude of the signal on line 32 and convert the 

20 amplitude during each sample period to a digital 
word representing the amplitude or intensity of the 
incoming analog signal. This stream of digital inten- 
sity words on bus 34 is read by a mapping com- 
puter 36. 

25 The purpose of the mapping computer 36 is to 

analyze the incoming digital data on bus 34 to 
determine one or more characteristics of the ana- 
log signal on line 32, This characteristic is mapped 
to a parameter of the MIDI interface, and a MIDI 

ao format data word or words are generated by the 
mapping computer 36. In the preferred embodi- 
ment, the Intensity of the signal on line 32 is 
mapped to the frequency of the musical note to be 
generated for that intensity word by the music 

35 synthesizer output device 38. In other embodi- 
ments, other characteristics such as the frequency 
of the signal on line 32 can be mapped to the note 
number (the note frequency) or the change in in- 
tensity or the change In frequency of the incoming 

40 analog signal may be mapped to corresponding 
changes in one or more parameters which may be 
specified in MIDI code to control the synthesizer 
38. 

in multi-channel embodiments, each channel 
45 may look for a different characteristic of the incom- 
ing analog signal in the same or In different pass- 
bands. Each characteristic may then have a sepa- 
rate mapping by the mapping computer 36 to a 
separate parameter in the MIDI format interface 
so code. Examples of MIDI format parameters which 
may be controlled are "note on", "note off", "note 
number", control parameters and system exclusive 
messages. The complete MIDI specification is in 
the public domain, A form of MIDI which is used in 
S3 the preferred embodiment. 

MIDI is an acronym for Musical Interinstrument 
Digital Interface. This interface enables synthesiz- 
ers, sequencers, home computers, rhythm ma- 
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chines and so on to be interconnected through a 
standard Interface. There are standard MID! inter- 
face hardward circuits and standard MIDI data for- 
mats. MIDI communication is achieved through 
multi-byte messages consisting of one status byte 
followed by one or two data bytes. There are 
sixteen available channels for data transmission of 
MIDI code words between transmitters and receiv- 
ers. The contents of the MIDI interface specification 
are hereby incorporated by reference. 

The music synthesizer 38 receives the MIDI 
format data words on the output bus 40 from the 
mapping computer and generate musical notes In 
accordance with the MIDI data. In the embodiment 
of Figure 2, the preamplifier 24 and the filtering 
circuit 26 perform the same functions that the cor- 
responding circuits perform in the embodiment of 
Rgure 1. Likewise for the digital converter 28. The 
mapping computer 38 in the embodiment of Figure 
2 does a different mapping than the mapping com- 
puter in Figure 1 , however, since the SMPTE inter- 
face format is used. In the system of Rgure 2. the 
output device is a video game 40. This video game 
drives a display 42. Video game 40 may be any 
video game. In the application of Figure 2, the user 
inputs will arrive as code words which may be 
converted to signals which control the operation of 
the video game. In other words the digital data may 
be converted to the signals ordinarily read by the 
video game from the user's joystick, fire control 
button, etc. In some embodiments the SMPTE In- 
terface format may be used. The SMPTE interface 
format is well known to those skilled in the video 
art. 

The functioning of the mapping computer 38 
will be to take the stream of digital data on the bus 
34 and map it according to some function data 
which supplies the necessary user input data to 
control the video game 40. The particular mapping 
function depends upon the particular type of input 
from the user that is required by the video game 
40. Suitable interface circuitry for the video game 
40 will take the user input from the mapping com- 
puter 38 via bus 44 and convert it to the appro- 
priate signals to control the action of the video 
game 40. 

In the application of Figure 2, the user may 
provide input to the video game 40 by thinking 
certain thoughts, making certain eye movements or 
performing certain movements of various muscles 
in the user's body, depending upon where the 
transducers 12, 14 and 16 are placed. Many suit- 
able video games are available to serve as the 
output device In the application of Rgure 2. Those 
skilled in the art will be able to easily design 
appropnate digital to analog conversion interface 
circuitry to accept the interface data on bus 44 and 
convert it to the necessary control signals for the 



video game 40. 

The programming of the mapping computer 36 
will depend upon the type of user interface imple- 
mented by the video game 40. Since most video 
s games require positional control inputs and fire 
control inputs, the mapping computer 36 will gen- 
erally be programmed to map the digital data on 
bus 34 to the appropriate position control and fire 
control data words on bus 44 to control the video 

70 game 40. Of course for other types of video games 
which require other types of user input, the map- 
ping computer 38 would be programmed to supply 
the appropriate type of user input required by the 
video game 40. 

75 Figure 3 shows a process of collecting elec- 

trical signals from a living body and analyzing 
these signals for certain characteristics and for 
generating output manifestations in real time based 
upon these characteristics. The process consists 

20 essentially of an input step wherein biologically 
related electrical signals such as EEG, EMG, EOQ. 
or EKQ signals are sensed via electrical trans- 
ducers In a known manner. This step is symbolized 
in block 46. Step 46 also symbolizes the process 

25 of filtering or otherwise processing the analog sig- 
nals collected from the body to derive the desired 
electrical signals where those electrical signals 
may be obscured by noise or by other signals 
generated in the body which are not of interest. 

30 Block 48 represents the steps in the process of 

converting the analog signals derived In step 46 to 
digital data. Block 48 can represent a straight 
analog-to-digital conversion of intensity values to 
corresponding digitaJ values or other types of pro- 

3S cessing. For example, the frequency of the elec- 
trical signals may be converted to an analog signal 
and the amplitude of that analog signal then con- 
verted to a corresponding digital value. Likewise, 
the changes in frequency or changes in Intensity of 

40 the analog signals of interest may be derived and 
then converted to analog signals which have am- 
plitudes corresponding to the change in frequency 
or change in intensity. The amplitude of these 
signals may then be converted to corresponding 

45 digital data. The particular process that block 48 
represents depends on the application and the 
characteristics of interest of the electrical signals 
sensed In step 48. 

Block 50 represents the process of analyzing 

so the digital data generated in step 48 for one or 
more characteristics. In other words, block 50 re- 
presents the process of determining from the digi- 
tal data representation of the electrical signals gen- 
erated by tiie body certain characteristics of those 

S6 electrical signals generated by ttie body. For exam- 
ple, step 50 may represent the averaging of the 
digital data words over a particular interval repre- 
senting intensity, changing intensity, frequency or 
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change in frequency. Other forms of prccassing 
may also be used such as digital filtering, convolu- 
tion, correlation, or other well known digital signal 
processing techniques. 

Block 52 represents the process of mapping 
the characteristic derived in step 50 to digital data 
which controls a particular parameter or parameters 
of the output manifestation. In the preferred em- 
bodiment where the output device is a music syn- 
thesizer, block 52 represents the process of map- 
ping the Intensity of the incoming signals to the 
frequency of musical notes that are generated. For 
other embodiments where different parameters of 
the synthesizer are to be controlled, block 52 re- 
presents the particular mapping which Is desired. 
For example, the frequency of the incoming analog 
signal may be mapped to the harmonic generation 
by the music synthesizer or to the program num- 
ber. Or, changes in intensity of the incoming signal 
may be mapped to a conresponding attack rate or 
decay rate of the musical notes generated. Any 
one of a number of different characteristics about 
the incoming electrical signal may be mapped to 
one or more parameters of the output manifestation 
which may be controlled. 

Block 54 represents the process of generating 
output control signals or output digital data to Im- 
plement the mapping performed by the process 
step symbolized by block 52. 

Block 56 represents the process of generating 
musical notes or other output manifestations in real 
time based on the digital data or control signals 
generated by the process steps of block 54. 

Although the process depicted in Figure 3 is 
very general, the teachings of the Invention enco'm- 
pass a wide variety of emiDOdlments. There are 
many options as to the types of electrical signals 
which may be sensed from the living body, the 
manner in which those signals are converted to 
digital data and the manner in which that digital 
data is analyzed. Further, there are many options 
as to the form of mapping which may be Imple- 
mented by the steps symbolized by block 52 and 
as to the type of output data generated from the 
mapping process. Also, the type of output device 
which is used is very application dependent The 
specifics of any one of the steps mentioned above 
will therefore depend upon the particular applica- 
tion in which the teachings of the invention are to 
be employed. There follows a discussion of a spe- 
cific embodiment which illustrates the manner in 
which the teachings of the invention may be em- 
ployed in a particular application. 

Referring to -Rgure 4, there is shown a block 
diagram of a specific system which employs the 
teachings of the -invention to map the intensity of 
incoming biologically generated electrical signals to 
musical notes which have a corresponding fre- 



quency. The circuitry within block 60 represents 
typical circuitry for one channel of sensing appara- 
tus to sense the electrical signals generated in the 
human body. Multiple channels may be used. Fun- 
5 damentaily, each channel sensing circuit consists 
of various gain and filtering stages to process the 
analog signal to remove noise and unwanted signal 
components prior to analog-to-dlgital conversion. 
Three transducers 12, 14 and 16 are used for 
10 each channel. Transducers 12 and 16 are coupled 
to the positive and negative inputs of a differential 
amplifier 62 which has its signal ground connected 
to a third transducer 1 4. 

The transducers are electrical contacts which 
15 are applied to the skin of the subject in a position 
to sense the desired signal. For example, if EKQ 
signals are to be detected, the transducers will be 
taped to the skin of the chest of the subject In a 
known manner to detect the electrical signals gon- 

30 erated by the heart muscles. 

The differential amplifier 62 Is a quiet amplifier 
with low internal noise generation and a high com- 
mon mode rejection. 

The output of the differential amplifier 62 is 

23 coupled to the input of a high pass filter 64. The 
purpose of the high pass filter Is to eliminate the 
D.G. offset which Is present on the output signal 
tine 66 of the differential amplifier 62. The high 
pass filter has a roll off frequency of 1 Hz to 

30 accomplish this function. 

The output of the high pass filter 64, in some 
embodiments, is coupled to the positive input of a 
single-ended amplifier 68. This amplifier Is used as 
a variable gain stage. The negative input of the 

35 amplifier Is coupled to the wiper arm of a poten- 
tiometer 70 which has one end of the resistive 
element coupled to ground and the other end 
coupled to the output of the differential amplifier 
68. The purpose of this amplifier is to increase the 

40 amplitude level of the electrical signal detected 
from the body by a user selectable amount. The 
output of the variable gain stage 68 Is coupled to 
the input of a low pass filter 72. 

Low pass filter 72 has a 1 kHz roll off point 

43 such that only signals having frequencies less than 
1 kHz can pass through this filter without substan- 
tial attenuation. Collectively, the filters 64 and 72 
represent a band pass filter which passes fre- 
quencies higher than 1 Hz and lower than 1 kHz. 

50 These roll off frequencies on either end of the 
passband may be adjusted for the frequency of 
interest. For example, if alpha waves in an EEG are 
to be detected, the filter roll off points may be 
adjusted to pass only the band of frequencies 

55 around 10 Hz in which the alpha waves are found. 

A multiplexer 74 is needed if more than one 
channel is present. The multiplexer has numerous 
inputs, each of which is connected to an output of 
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ona of XhQ channels and. specifically, to the low 
pass fitter 72 output In one of the channels. Mul- 
tiplexer 74 also has a plurality of select inputs 
which are coupled to a select bus 76. This bus is 
coupled to the output of an address latch 78 which 
has its data inputs coupled to the address bus 80 
of a signal-processing chip 82. In preferred enn- 
bodiments. a 320C10 is used. The load control 
input of the address latch 78 is coupled to the 
ADDRESS VALID control signal on line 84 from the 
signal-processing chip. The signal-processing chip 
therefore controls switching by the multiplexer 74 
by writing the address of the channel whose input 
signal Is to be analyzed on the address bus 80 and 
latching it into the address latch 78 by activating 
the ADDRESS VAUD signal on line 84. This 
causes a specific bit pattern on the bus 76 which 
causes the multiplexer 74 to select the input signal 
line from the desired channel for coupling to an 
output signal line 88. 

The selected analog signal on line 88 is coup- 
led to the Input of an analog-to-digital converter 90. 
The A/ D converter receives a clock signal from the 
signal processing chip 82 via a clock divider 91 
and a line 93. Each upward transition of the clock 
signal on line 93 turns on the analog-to-digital 
converter 90 to start the conversion process for the 
analog signal at the input of the A/D converter. 
When the analog-to-digital converter 90 has com- 
pleted its conversion process, it activates the 
DONE signal on line 94. This signal is latched into 
a latch 98 acting as a status flag. This latch has its 
output coupled to the signal-processing chip 82 by 
the signal fine 98. Signal line 98 can be coupled to 
the interrupt input of the signal-processing chip 82 
in some embodiments or it may represent the 
process of polling the flag latch 96 from time to 
time to determine whether the conversion Is done. 

When the signal-processing chip senses the 
DONE signal, it activates a READ signal on line 92. 
This signal causes the A/D converter to place the 
conversion data on the data line 104. The clock 
divider 91 receives a clock signal from the signal- 
processing chip 82 and divides it by a factor of 
250. The resultant slower clock signal is output on 
line 93 to the "start conversion" input of the 
analog-to-digital converter 90- 

The signal-processing chip 82 controls the 
multiplexer 74 and the analog-to-digital converter 
90 in such a way as to sample the input signals 
coming from each channel at a rate which is set by 
the Nyquist criteria for the expected frequency of 
the signal' on that channel. The Nyquist criteria 
requires that for effective sampling of a signal of 
frequency F with no aliasing, the sampling fre- 
quency must be at least 2F, In the application 
shown in Rgure 4. the frequencies of interest are in 
the frequency band from 1 Hz to 1 kHz. Thus, the 



signal-processing chip 82 controls the sampling of 
the signal from channel 1 such that the effective 
sampling rate is 2 kHz. 

The output digital data from the analcg-to-digi- 

s tal converter 90 is temporarily latched in a buffer 
100. The buffer 100 stores the digital data until 
such time as the signal-processing chip 82 does an 
I/O transaction to read the contents of the buffer via 
data bus 102. The READ signal on control line 92 

10 from the signal-processing chip is also coupled to 
the load control input of the buffer 100. Each time 
the READ signal is activated, the anaiog-to-digital 
converter 90 starts a new conversion and the buffer 
100 is loaded with the data from the previous 

?s conversion then present on the data output bus 104 
of the analog-to-digital converter 90. 

The signal-processing chip gathers digital data 
from each channel in the manner described above. 
This data is then analyzed and mapped to digital 

20 data having MIDI word format for transmission to 
the output device via a UART 106. The UART has 
Its parallel data input coupled to the data bus 102 
of the signal-processing chip 82. The signal-pro- 
cessing chip does I/O transactions with the UART 

25 106 to write parallel data therein. The UART then 
converts this parallel data to serial format M\D\ 
words and transmits the resultant MIDI word stream 
on a serial format signal line 108 to the syn- 
thesizer. 

30 The signal-processing chip 82 performs its 

data gathering and mapping function in accordance 
with the flow chart shown on Figure 5. Referring to 
Figure 5. there is shown a flow diagram of the 
preferred embodiment of the invention for an ap- 

35 plication where the intensity of the electrical signals 
generated in the body are mapped to the fre- 
quency of the notes generated by the music syn- 
thesizer. The first step in this process is to reduce 
the effective sample rate as symbolized by block 

40 110 since many more samples have been taken by 
the sampling circuitry than are necessary to repre- 
sent the narrow bandwidth of the biopotential. The 
lower effective sample rate allows more time for 
calculation between samples. That Is. because the 

45 music synthesizer can accept MIDI words at a far 
slower rate than digital data may be acquired from 
the body, it is important to slow the effective sam- 
pling rate down to a level at which MIDI words can 
be generated and accepted by the music syn- 

50 thesizer in order to generate musical notes at an 
acceptable rate, in the preferred embodiment, this 
is done by the process steps symbolized by block 
110. This block symbolizes the process of accept- 
ing every sixth sample which is input from the 

65 analog -to-digital converter 90. This gives an effec- 
tive sample rate of 2 kHz from the actual sample 
rate of 12 kHz at which the multiplexer 74 and the 
analog-to-digital converter 90 are being driven. 
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In order to establish an inltia) collection of data 
samples from which the mapping process may 
start, a step 112 is performed. This step represents 
the process of loading the first 8 samples into a 
memory array for purposes of averaging. The aver* 
age Intensity over a group of samples is used for 
the mapping function rather than Individual sample 
values to eliminate the effects of noise in the 
system and to get smoother transitions in the flow 
of musical notes which are generated. 

The next step is to average the sample values 
in the array as symbolized by block 114. On the 
first time through the program loop, there will be 
only 8 samples in the array, and the average will 
be over only these 8 samples. However, the array 
has the capacity to store 32 samples, and In sub- 
sequent loops through the step 114, the average 
will be taken over 32 samples because new sam- 
ples will have arrived during the time the loop is 
being traversed* 

After the average of the values in the array is 
computed, a test 116 is performed to determine 
whether the average is greater than a predeter- 
mined threshold. The purpose of this test is to 
eliminate noise from the mapping process such 
that noise samples do not create musical notes. If 
the average is less than the given threshold, a step 
118 is performed where the signal-processing chip 
waits one sample time and then branches back to 
step 1 1 4 via path 1 20. 

If the result of the test of step 116 Is that the 
average is greater than the threshold value, then 
step 12t is performed. Step 121 represents the 
process of performing an I/O transaction to the 
UART 106 in Figure 4 to send a MID! word which 
is designated "note on". This tells the output music 
synthesizer that the signal-processing chip 82 de- 
sires to turn on a note. However, the particular note 
which is to be turned on must also be sent in 
another MIDI word before the synthesizer can turn 
on a note. To communicate the particular note that 
the signal-processing chip wishes to turn one, step 
1 22 Is performed. Step 1 22 represents the process 
of waiting four sample times (at the effective sam- 
ple rate) and then sending the note number MIDI 
word to control which note is played by the music 
synthesizer. 

The mapping process of mapping biopotential 
intensity (amplitude) to note number is carried out 
in step 122. This is done by taking the computed 
average and using that as the note number. This 8 
bits a data is put on data lines D8 through 015 of 
the signal-processing chip data bus coupled to the 
UART 106. In other embodiments a look up table 
could be consulted or some mathematical algo- 
rithm could be performed. 

The third MIDI word in a "not© on" MIDI se- 
quence is the "note on velocity"; in other words, to 



play a musical note, the music synthesizer must 
receive three MIDI words, one being the "note on" 
command, one being the "note number" command 
word to identify the frequency of the note desired, 

s and the third being the "note on velocity" which 
determines part of the feel or impression of the 
note. Step 124 represents the process of sending 
the "note, on velocity" after waiting four sampie 
times. In the algorithm shown In Rgure 5. the "note 

10 on velocity" is fixed at a value half way between 
the minimum and maximum possible MIDI note on 
velocity values. In other algorithms, the "note on 
velocity" word can be generated by mapping from 
the data on another channel such as an EEG or 

IS EKQ channel. 

After the "note on" MIDI sequence is sent, the 
note will be allowed to sound for 8 sampie times. 
After 8 sample times, the "note off" MIDI sequence 
is sent. The first step In this sequence is symbol- 

20 ized by block 126. This step represents an I/O 
transaction to write the "note off" command word 
into the UART. 

Block 128 represents the process of sending 
the "note number" MIDI word to the output device 

25 to teil It which note to turn off. 

The final step in the "note off" sequence is 
symbolized by block 130 which represents the 
steps of waiting four sample times and sending the 
"note off velocity" MIDI word. This word estab- 

30 lishes part of the feel or impression created by the 
note. 

It is useful to have musical manifestation of 
EEG output symbolizing brain waves. The appara- 
tus of Figure 4 can be used to detect the alpha 

35 wave component (or other aspects) of brain waves 
by adjusting the low frequency roll off of the high 
pass filter 64 to be approximately 7 Hz and adjust- 
ing the high frequency roli off of the low pass filter 
72 to be approximately 30 Hz. Although it is not 

40 mandatory, it Is preferred to move the variable gain 
stage 68 to the output side of the low pass filter 72. 
The only other changes necessary are to change 
the programming of the signal-processing chip 82 
to implement the method shown in Figure 6. 

45 The first step in the EEG mapping process is 

the MIDI set up step. This step is -common to all 
the mapping programs disclosed herein but has not 
been shown in all the flow charts for the sake of 
brevity. This MIDI set up step represents the pro- 

50 cess of sending the MIDI program code to the 
synthesizer to establish a set of MIDI parameters 
which control the characteristics of the sound. Each 
set of such parameters has a program number. 
This effectively establishes the synthesizer as hav- 

ss ing a certain characteristic sound. 

To perform mapping of the alpha wave compo- 
nent of a brain wave, it is first necessary to find the 
amplitude of the alpha wave component. Alpha 
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waves have an average characteristic frequency of 
10 Hz. Since the Input signal, after fiftering, has 
frequency components from 7 Hz to 30 Hz, the 10 
Hz component must be separated from all the 
other frequencies in some manner. A fast fourier 
transform, sometimes called an FFT, is well known 
in the art. and it will not be described in detail 
herein. An FFT is performed to separate the 10 Hz 
component from the rest of the signal. To perform 
on FFT on a digital computer, a discrete sample 
set Is required. Step 142 represents the process of 
collecting the number of samples required to per- 
form the FFt. In the preferred embodiment. 8 sam- 
ples are used for the FFt. although in other em- 
bodiments larger numbers of samples may be 
used. 

In the preferred embodiment, the effective 
sampling rate for EEG mapping is 150 Hz. Since 
the alpha wave component is only 10 Hz, any 
sampling rate of 20 Hz or greater will suffice for 
purposes of practicing the invention. 

The next step in the process is symbolized by 
block 1 44 which represents the steps of performing 
the FFT calculation. The FFT determines the am- 
plitude coefficients of the FFt fourier series for a 
series of harmonics which, when mixed together at 
the amplitude specified by their coefficients, will 
combine to create the original signal. In other 
words* the fourier transform is a transformation 
from the time domain to the frequency domain. It 
.yields the frequency components in the frequency 
domain of any signal in the time domain which is 
subjected to the fourier transform mathematical op- 
eration. The purpose of performing the FFt in the 
method of Figure 6 is to determine the amplitude 
of the 10 Hz component in the input signal repre- 
sented by the samples collected in step 142. Step 
146 represents the examination of the results of the 
FFt of step 144 to determine the 10 Hz coefficient 
value. This coefficient represents the amplitude of 
the alpha wave component. 

Step 148 represents the process of comparing 
th.e 10 Hz coefficient value determined in step 146 
to a fixed threshold. This is done to prevent alpha- 
wave components of insufficient magnitude from 
causing changes in the mapping. 

If the 10 Hz component is less than the thresh- 
old value, then step 150 is performed. Step 150 
represents the process of waiting for a new batch 
of samples to be collected for purposes of perform- 
ing a new FFT transform. 

If the 10 Hz component is greater than the 
threshold, then a step 152 is performed. This step 
represents the process of sending a program 
change sequence of MIDI code words to the output 
device to cause the "voice" or sound of the music 
synthesizer to change. As is well known in the art, 
music synthesizers and other MIDI machines have 



prearranged programs or sets of MIDI parameters 
which cause the machine to have a characteristic 
sound, such as breaking glass, homs. drums, cym- 
bals, or other known or as yet unknown sounds. 

s Each of these sets of parameter values are iden- 
tified by a program number. In the method shown 
in Rgure 6, the subject's thought pattern can 
change the program when the alpha-wave compo- 
nent has a sufficient magnitude. When this occurs. 

10 step 152 Is performed to send a sequence of MIDI 
words which causes the program number to be 
changed. The sequence is only two words long, the 
first MIDI word being the "change program" com- 
mand and the MIDI "channel" on which the pro- 

75 gram Is to be changed, and the second word being 
the program number to which the MIDI machine is 
to be programmed. 

As is well known in the art of generating com- 
puter music, the MIDI Interface has sixteen chan- 

20 nets. Thus, complex music can be performed by 
sending streams of MIDI "note on" and "note off" 
sequences, control parameters and MIDI program 
numbers on each of the sixteen channels. The 
MIDI machine time-division multiplexes itself 

25 among these channels and plays all the notes in 
accordance with the program and other control 
information received on a particular channel. Since 
this appears to happen simultaneously to the user, 
very complex musical pieces can be generated in 

30 this manner. 

Of course, those skilled in the art will appre- 
ciate that other components of the brain wave may 
be sensed and mapped to different numbers such 
that multiple program changes may result -from 

35 changing thought patterns. 

Referring to Figure 7, there is shown a flow 
diagram of the process implemented by the signal- 
processing chip 82 in mapping EKG signals into 
MIDI code which is suitable to control a drum 

40 machine. Those skilled in the art will appreciate 
that a drum machine is not the only output device 
which could be controlled by the MIDI code gen- 
erated by the EKQ mapping routine of Rgure 7 
since the mapping function may be adjusted for the 

45 type of machine and parameter to be controlled. 

The EKG mapping routine starts out with step 
160, which is a MIDI set-up step to send the 
program number to the synthesizer or other MIDI 
machine to establish its sound quality. Other con- 
so trol parameters are also sent in MIDI code to 
properly set up the machine. 

The fundamental frequency of interest In EKG 
mapping is in the frequency range of 0.1 Hz to 4 
Hz. The intensity of the various components of the 

66 EKG signal is determined by the steps symbolized 
by block 162. This- process can be accomplished In 
any known way. such as by forming the FFT as 
was done in the EEG mapping of Figure 6, or by 
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digital or analog filtering out of a!l but the desired 
component or components and measurement of 
the intensity. Any known signal processing tech- 
nique to determine the intensity of the desired 
frequency components will suffice for the EKG 
mapping routine, and the same way be said for the 
EEG mapping routine of Figure 6, except that in 
that case the amplitude of the 10 Hz alpha-wave 
component of the brain wave is sought. 

Next, the intensity of the desired frequency 
component of the EKG signal is compared to a 
fixed threshold, as symbolized by test blocl< 164, If 
the amplitude is less than the threshold, then step 
166 is performed which is simply a process of 
waiting for another batch of samples to arrive to 
perform another FFT calculation. If digital or analog 
filtering is being used to separate out the desired 
frequency component, then step 166 is skipped, 
and control is transferred from step 164 directly 
back to step 162, and step 166 is not performed. 

If the amplitude of the desired frequency com- 
ponent is greater than the threshold, a step 168 is 
performed to send a "note on" sequence to. e.g.. a 
MIDI drum machine. Those skilled in the art will 
appreciate that other types of MIDI machines may 
be substituted, or that the mapping need not be 
from the desired frequency component intensity to 
a note number. For example, the mapping may be 
from the desired frequency component intensity to 
any other MIDI parameter such as program num- 
ber! control parameter, and/or system exclusive 
messagesi and so no. 

After step 168 is performed, a step 170 Is 
performed to send the "note off" sequence to the 
drum machine after a suitable delay. Those skilled 
in the art will appreciate that the delay may be any 
suitable delay set by the user. 

Referring to Figure 8. there is shown a flow 
diagram for a process to modify MIDI code by 
evaluating the body's reaction to musical notes 
where these notes were generated in response to 
biopotentials sensed in the body. Fundamentally, 
the process shown In Figure 8 evaluates biopoten- 
tials at time zero and generates musical notes 
therefrom. After the musical notes are generated, 
the body reacts to those notes, and biopotentials in 
the body change. These body potentials are 
reevaluated and new MIDI code is generated in 
response to changes in the biopotentials. This pro- 
cess is implemented starting with step 172, which 
senses the biopotentials In the manner described 
above using the apparatus described in Figure 4. 
or any other apparatus which is capable of sensing 
biopotentiars in the frequency range of interest 

Step 174 represents the process of evaluating 
the biopotentials for their characteristics and map- 
ping these characteristics to one or more of the 
various MIDI parameters. After this mapping has 



occurred, the appropriate MIDI code words are 
generated and transmitted to the output device. 

Step 176 represents the process performed by 
the output device in generating musical notes In 

5 response to the MIDI code generated in step 174. 
After these musical notes have been generated, the 
body wilt react to them. When the body reacts, the 
brain wave potentials change, and possible the 
heart rate or EKG potentials will change. Further, 

w the muscle (EMG) may also change if the body 
moves in response to the notes. Step 178 repre- 
sents the process of reevaluating the characteris- 
tics of these biopotentials to evaluate the changes 
therein in response to the musical notes generated 

;5 in step 176. Step 178 also represents the process 
of evaluating the new characteristics of the 
biopotentials so sensed and mapping those 
changes to changes in the MIDI code generated In 
step 174. 

20 Step 180 represents the process of generating 

new MIDI code in response to the mapping per- 
formed in step 1 78. Processing then returns to step 
172. 

Referring to Figure 9, there Is shown a flow 

25 diagram for a process of modifying incoming MIDI 
code from another MIDI machine in response to 
certain characteristics of biopotentials generated in 
a living body. There are many types of MIDI ma- 
chines currently available, and moe MIDI machines 

30 are appearing constantly. Many of these machines 
•have MIDI code outputs such that musical data 
generated in the machine may * be transmitted in 
MIDI word format to other machines. Step 182 
represents the process of receiving such MIDI 

35 code from another MIDI machine. Step 184 repre- 
sents the process of sensing biopotentials from a 
living body in real time. Any type of biopotential 
may be sensed, as described herein or in a fashion 
other than as described herein. 

40 Step 1 86 represents the process of evaluating 

the characteristics of the biopotential so sensed. 
This evaluation is done in real time. Once the 
characteristics of the biopotentials are determined, 
these characteristics are mapped to corresponding 

45 modifications to the MIDI parameters of the Incom- 
ing MIDI code. For example. If the sensed 
biopotential Is an EKG signal, a rising heart rate 
could be mapped into higher note numbers or 
changed rhythms on a rhythm machine. Step 188 

50 represents the process of modifying the incoming 
MIDI code in response to the biopotential char- 
acteristics determined in step 186. New MIDI out- 
put code is then generated and output. 

Referring to Rgure 10. there is shown a sys- 

55 tem according to the teachings of the Invention to 
sense four different kinds of biopotential signals 
and to receive MIDI code from another machine 
and generate new MIDI code in multiple channels. 
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The system maps characteristics of the biopoten- 
tials into MIDI code on multiple channels for con- 
trolling various parameters of an output device, 
such as a music synthesizer. The system is com- 
prised of four different front-end channels to sens© 
biopotentials through electrodes placed on the 
body of a subject. The four channels are an EMG 
channel, 200, an EEG channel, 202, and EKQ 
channel, 204, and an EOO channel, 206 (to sense 
eye position). Each of these channels is similar to 
the channel 1 circuitry shown in Rgure 4 at 60. 
However, each channel may have its own frequen- 
cy passband, to eliminate any ianalog components 
of the biopotential frequencies ' other than the fre- 
quencies of Interest. 

The system also Includes an Interface 208 for 
receiving MIDI code from another MIDI machine. 
This interface receives and stores Incoming MIDI 
code words in, for example, a FIFO buffer in a 
UART. When a MIDI code word is In storage, the 
interface 208 raises an interrupt flag on line 210 to 
cause the signal-processing chip 82 to read the 
newly arrived MIDI words via a data bus 212. This 
I/O transaction would bo performed by writing the 
address of the interface 208 on an address bus 
214 coupled to a decoder 216. This decoder would 
then activate the ENABLE-2 signal on line 218 to 
cause the Internal buffer on the Interface 208 to 
place its MIDI word data on the data bus 212. The 
signal-processing chip 82 then performs a read I/O 
transaction to obtain the incoming MIDI code 
words. The signal-processing chip 82 may also 
periodically poll* the interface 208 Instead of using 
an interrupt structure in other embodiments. In the 
preferred embodiment, the signal-processing chip 
82 time-division multiplexes Its services among the 
four biopotential channels and the interface 208. 
That is. the signal-processing chip 82 sequentially 
generate the addresses of each of the biopotential 
channels and the interface 208 on the address bus 
214. The decoder 216 then sequentially generates 
appropriate select control signals on a bus 220 
coupled to the select control input of a multipJexer 
74 to cause it to sequentially couple each of the 
four outputs from the biopotential channels In turn 
to the output signal line 88 through the multiplexer 
74. 

An analog-to-digital converter 230 receives the 
input analog data and converts it to digital data 
which Is written via a bus 232 into a buffer 100. As 
each channel is selected by the signal-processing 
chip 82 via the multiplexer 74. the digital data 
corresponding to the sample from that channel is 
stored in the buffer 100. The signal-processing chip 
then does an I/O transaction to read the data in the 
buffer 100 by writing the address of the buffer on 
the address bus 214, thereby causing the decoder 
216 to activate an ENABUE-I signal on a line 234. 



This causes the buffer 100 to place its data on the 
data bus 212, and the signal-processing chip then 
roads it via an I/O read transaction. The signal- 
processing chip 82 then stores this data in an 

s Internal register or scratchpad RAM. or In external 
RAM 236 for processing. 

The mapping function the signal-processing 
chip 82 performs is highly application-dependent. 
Suffice It to say that the teachings of the invention 

70 contemplate that any mapping performed in real 
time between a characteristic of a biopotential 
sensed in real time to a parameter which may be 
controlled via a MID) word of a parameter which 
may be controlled In some other output device will 

J5 be within the scope of the invention. The invention 
is not limited to the MIDI interface. Any output 
devices which have parameters which may be con- 
trolled by the user will be within the scope of the 
Invention providing there is no prior art which re- 

20 stricts the scope of the Invention to something less. 
For example, it a machine is developed in the 
future which can alter the moods of a subject by 
causing electrical stimulus of the brain or otherwise 
altering the brain chemistry in some fashion, it is 

as within the scope of the invention to map the 
biopotential characteristics to parameters of such a 
machine which may be controlled to alter the mood 
of the subject by muscle movements or thought 
patterns. 

30 Signal-processing chip 82 has a service routine 

for each time slot. The type of mapping routine 
performed during each time slot depends on the 
particular type of channel data which is being pro- 
cessed during that time slot. For the EMG channel 

35 200. the mapping routine would be comprised of 
the steps shown In Figure 5. When EEG channel 
data is being processed, the mapping routine 
would be similar to the process shown in Rgure 6. 
When EKG data is being processed, the mapping 

40 routine would be similar to the routine shown in 
Figure 7. When dipole data from the eye from the 
EOG channel 208 is being processed, the mapping 
routine performed by the signal-processing chip 82 
would be similar to the mapping routine shown in 

4S Figure 5. When the signal-processing chip 82 Is 
processing incoming MIDI code from another ma- 
chine, the mapping routine would be similar to 
Rgure 9. 

The MIDI interface code contemplates the use 
50 of up to sixteen channels. Thus, sixteen separate 
MIDI word data streams may be sent to the same 
device in separate time slots. Thus, the output 
device may respond virtually simultaneously to up 
to sixteen different MIDI command word streams 
55 and appear to be simultaneously responding to ail 
of them. In reality, the output device is separately 
responding in sixteen different time slots to sixteen 
different command word streams. However, since 
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the switching Is so rapid, the human ear and brain 
would tend to integrate the distinct sounds into one 
composite aura! work. To accomplish this time- 
division multiplexing on the outputs, the signal- 
processing chip 82 would output the MIDI code 
generated during each time slot service routine to 
the output device on the corresponding MiOl chan- 
nel. This would be done by writing the data for 
each time sfot, i.e.* channel to the appropriate 
UART. Each UART can handle up to 16 MIDI 
channels, the system can control as many output 
devices as there are UARTs. These UARTs are 
shown symbolically at 240. The appropriate output 
device can be driven by writing the address of the 
appropriate UART on the address bus. This causes 
the decoder 216 to activate the appropriate enable 
signal on the ENABLE-2 bus 242 to cause the 
UART for the corresponding output device to be- 
come enabled. The signal-processing chip 82 
would then perform an I/O write transaction to the 
appropriate UART via the data bus 212. All the 
other UARTs in the collection of UARTs shown at 
240 would have their inputs coupled to the data 
bus 212 in a high-Impedance mode such that the 
Incoming data would be stored only by the UART 
that is enabled. That UART would then convert the 
parallel format data from the data bus 212 into a 
serial stream of MIDI code words which would be 
output to a music synthesizer or other output de- 
vice 244 via a serial data link 246. 



Claims 

1 . An apparatus for generating data for control- 
ling the generation of musical notes comprising: 
means for sensing biopotential signals In real time; 
means for receiving said biopotentials and mapping 
them in real time into MIDI code. 

2. An apparatus as defined in claim 1 further 
comprising any known machine capable of receiv- 
ing and utilizing MIDI code to control the oper- 
ations of said machine coupled to receive said 
MIDI code for generation of musical notes there- 
from. 

3. An apparatus for real time generation of 
digital data to control the generation of musical 
notes mapped from electrical signals generated in 
a living body comprising: 

at least one output device for converting incoming 
digital data for the control of musical notes into 
musical notes; 

means for sensing in real time at least one elec- 
trical signal generated in a living body and convert- 
ing said electrical signal in real time into a first 
stream of digital data samples representing said 
electrical signal; 

mapping means for receiving said first stream of 



digital data samples and for extracting in real time 
from said first stream of digital data samples at 
least one characteristic of said electrical signal 
generated in said living body and for mapping in 
5 real time each said characteristic and the changes 
therein into a second stream of digital data capable 
of controlling the generation of musical notes and 
for transmitting same to said output device. 

4. The apparatus of claim 3 wherein said 
10 means for sensing includes a differential amplifier 

having an output and having an input coupled to 
sense and to amplify said biological signal via two 
sensing electrodes and a ground electrode coupled 
to said inputs, said electrodes for being placed in 
T5 contract with a body, said differential amplifier hav- 
ing sufficient common mode rejection to substan- 
tially eliminate common mode noise at said input. 

5. The apparatus of claim 4 further comprising 
a bandpass filter means coupled to the output of 

20 said differential amplifier for passing to said map- 
ping means without substantial attenuation only fre- 
quencies within the passband of said bandpass 
filter means. 

6. The apparatus of claim 5 wherein said sens- 
25 ing means further comprises a plurality of data 

channels each of which includes a differential am- 
plifier and a bandpass filter, at least some of said 
data channels having different passbands for said 
bandpass filters, each said channel for sensing a 
00 different biopotential signal generated In a living 
body and each said channel having a signal output. 

7. The apparatus of claim 6 further comprising 
a multiplexer having a plurality of inputs each of 
which Is coupled to said signal output of one of 

35 said channels, and having a signal output coupled 
to supply said electrical signal to said mapping 
means, and said multiplexer having a control input 
for controlling which input is coupled to said signal 
output, said control input coupled to said mapping 

40 means to receive select signals therefrom, and 
wherein said mapping means Is also for generating 
signals to control which said, input said multiplexer 
couples to said output. 

8. An apparatus for converting biological elec- 
45 trical signals to musical notes comprising: 

Input means for sensing and amplifying electrical 
signals generated by a living body and for filtering 
out unwanted components of said electrical signals 
and for converting the remaining components to a 

50 first stream of digital datar 

processing means for receiving said first stream of 
digital data and extracting from it one or more 
characteristics of said electrical signals and map- 
ping in real time each said characteristic and its 

55 changes to a MIDI parameter and corresponding 
changes in said parameter, and for generating a 
stream of MIDI words for each said characteristic; 
and 
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at [east one MIDI machine for receiving said stream 
of MIDI words and for converting said MIDI words 
to musical notes or other musical data. 

9. An apparatus for converting electrical signals 
generated in a body into video data in real time 
comprising; 

input means for sensing a biological electrical sig- 
nal and for converting it In real time to a first digital 
data stream; 

a video output device for converting incoming digi- 
tal data containing user input Information into video 
display Information; and 

mapping means for receiving said first digital data 
stream and for extracting a characteristic of said 
biological electrical signal embodied in said first 
digital data stream and for mapping said char- 
acteristic into a second digital data stream capable 
of supplying said user input information and for 
transmitting same to said output device. 

10. An apparatus for modifying MIDI code gen- 
erated by a machine in accordance with biopoten- 
tials generated by a living body comprising: 
means for receiving MIDI code from another ma- 
chine; 

mapping means for sensing electrical signals gen- 
erated by a living body; 

mapping means for determining a characteristic of 
said electrical signal and mapping the characteris- 
tic to a change to be made in said received MIDI 
code; and 

means coupled to receive said MIDI code from 
said other machine and data from said mapping 
means regarding the change to be made In said 
MIDI code and for making the change required by 
said data from said mapping means and for output- 
ting MIDI code modified in accordance with signals 
from said mapping means. 

11. An apparatus for mapping one or more 
frequency components of brain waves to MIDI code 
comprising: 

means for sensing brain waves and for filtering out 
frequencies outside a passband including the char- 
acteristic frequency .of the desired frequency com- 
ponent and for outputting the filtered output signal; 
means for sampling said filtered output signal at a 
rate which is sufficiently high to prevent aliasing; 
means for performing a fourier transform operation 
on a plurality of said samples; 
means for determining the amplitude of the desired 
frequency component coefficient resulting from 
said fourier transform operation at the characteristic 
frequency of said desired frequency component; 
means for mapping the Intensity of the desired 
frequency component Into MIDI code. 



12. The apparatus of claim 13 wherein said 
means for mapping includes means for mapping 
changes in intensity of the alpha wave component 
of brain waves to changes in the program number 

5 of the MIDI code. 

13. An apparatus for generating MIDI code 
from electrical signals generated In a human body 
comprising: 

means for sensing electrical signals generated in a 
10 muscle or group of muscles In a living body and 
for generating a first output signal; 
means for sensing brain waves generated in a 
living body and for filtering out unwanted frequency 
components and for generating a second output 
15 signal; 

means for sensing electrical signals generated in a 
living body to drive a heart muscle and for filtering 
out unwanted frequency components and for gen- 
erating a third output signal; 

20 means for sampling said first second and third 
output signals at sample rates which are high 
enough to prevent aliasing and for generating digi- 
tal data representative of each of said first, second 
and third output signals; and 

25 processing means having an input for receiving 
said digital data and having an output, said pro- 
cessing means for extracting a characteristic of 
each of said first, second and third output signals 
from said digital data and for mapping each said 

30 characteristic to a different channel of MIDI code 
and for outputting each said channel of MIDI code 
at said output. 

14. The apparatus of claim 13 wherein each 
channel of MIDI code is output during a different 

35 time interval at said output. 

15. ' A method of generating MIDI code com- 
prising the steps of: 

sensing electrical signals generated in a living 
body; and 

40 mapping said electrical signals to MIDI code. 

16. A method of generating digital code for 
controlling a video game comprising the steps of: 
sensing electrical signals generated in a living 
body: and 

45 mapping said electrical signals to digital code suit- 
able for supplying user input data for a video 
game. 

17. A method of generating MIDI code com- 
prising the steps of: 

50 sensing electrical signals generated in a living 
body and filtering out unwanted frequency compo- 
nents: 

converting said filtered electrical signals to digital 
data; 

55 extracting at least one characteristic of said elec- 
trical signals from said digital data: and 
mapping said characteristic of said electrical sig- 
nals into MIDI code. 
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18. The method of claim 17 further comprising 
the step of generating musical notes from said 
MIDI code. 

19. A method of generating MIDI code from 
brain waves comprising the steps of: 

sensing electrical brain waves In real time; 
sampling the brain waves at a sample rate which Is 
at least high enough to prevent aliasing and con- 
verting the samples to digital data: 
performing a fourier transform on the digital data of 
a plurality of said samples: . 
determining the magnitude of the fourier series 
coefficient for the frequency component of interest; 
and 

mapping the magnitude of the fourier series coeffi- 
cient of the frequency component of interest into 
MIDI code. 

20. The method of claim 1 9 further comprising 
the step of filtering out frequency components out- 
side a passband of frequencies wherein the char- 
acteristic frequency of desired frequencies is within 
said passband prior to performing the sampling 
step, and wherein the step of determining the mag- 
nitude of the coefficient of the frequency compo- 
nent of interest comprises determining the mag- 
nitude of the fourier series coefficient for the fre- 
quency component having the characteristic fre- 
quency of alpha waves such as beta, delta and 
theta waves. 

21. The method of claim 19 further comprising 
the step of filtering out frequency components out- 
side a passband of frequencies wherein the char- 
acteristic frequency of EKQ waves is within said 
passband prior to performing the sampling step» 
and wherein the step of determining the magnitude 
of the coefficient of the frequency component of 
interest comprises determining the magnitude of 
the fourier series coefficient for the frequency com- 
ponent having the characteristic frequency of EKG 
waves. 

22. The method of modifying MIDI code com- 
prising the steps of: 

sensing electrical signals generated In a living 
body; 

mapping said electrical signals to MIDI code: 
using said MIDI code to generate musical notes; 
reevaluating said electrical signals generated in 
said living body for the effect said musical notes 
had on said electrical signals; and 
modifying said MIDI code by mapping the changes 
in said electrical signals to corresponding changes 
in said MIDI code. 

23. A method of modifying MIDI code received 
from another machine capable of generating MIDI 
code comprising the steps of: 

sensing electrical signals generated In a living 
body; 

receiving said MIDI code from another MIDI ma- 



chine; 

evaluating the characteristics of said electrical sig- 
nals and mapping said characteristics to changes 
in said MIDI code; and 
5 changing the MIDI code in accordance with the 
changes determined in said mapping step. 
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